This disclosure relates to secure, automated, augmented reality, and interactive control, operation, maintenance, system engineering, and configuration management apparatuses and methods that can wirelessly interact with data libraries as well as engineering and maintenance activities at a different location and employs wearable interface systems. In particular, aspects of embodiments of the invention can include wearable electronic systems that provide an augmented reality system which generates a wearable visual interface for facilitating a variety of tasks, employs machine vision pattern recognition (e.g., shape or bar code associated with selected equipment), and includes variety of user command input interfaces (e.g. gesture recognition, voice command, and/or other interfaces e.g., a wireless keyboard or digital stylus). Apparatuses and methods can include support for tasks such as a configuration management task, e.g., visual recognition and overlays of stored engineering drawings to support verification of technical data (e.g., technical data verification based on machine vision comparisons with stored three dimensional model based engineering technical drawings and associated data), equipment calibration tasks supported via visual and voice command interfaces, visual overlays of equipment or subassemblies showing operational or maintenance status information from remote data servers, etc.
Control and maintenance of multiple fielded systems in remote locations or systems without physical access poses a significant logistics and human systems challenge. For example, remote location maintenance or control can require changes of configurable parameters in subsystems of systems that frequently require on-site personnel and technical data libraries. Some efforts have been made to provide remote control and maintenance capabilities. However, existing systems are not integrated into visualization technology for a user to monitor and change selected configurable parameters within subsystems of a system and be able to connect to a fleet of systems in an operation location. Existing systems are also vulnerable to system intrusion and interruption the performance of remote maintenance function or operations due to inadequate or lack of cybersecurity such as user authentication and encryption of data.
In one embodiment of the present invention, a user with a head mounted device (HMD) can be positioned with respect to a system under test with various embodiments of the invention. Another embodiment can incorporate an object with a marking designator to reference to a group of systems and their subsystems. Image recognition and augmented reality technology can be used to identify the system under test and generate one or more three-dimensional models of a system and associated subsystems with configurable parameters. Voice and gesture recognition can be incorporated to initiate a command to enable remote calibration. Authentication system and encryption methods can also be included to provide cybersecurity functionality to use various embodiment of the invention in an operational environment where data intrusion is expected. Additional elements can include remote transmission to a remote server with an authentication service and data library as well as a capability to enable operation and data retrieval from the remote server as well as facilitating support personnel at different locations to interact with on-site equipment and personnel in a variety of ways.
In general, aspects of the invention can include a secured remote maintenance, configuration management, and systems engineering apparatuses and methods including wearable augmented reality interface systems. Embodiments can include machine vision visual recognition, multiple command interfaces, location and orientation system, wireless communication system coupled with one or more remote data sources, interactive remote maintenance support, fault troubleshooting, maintenance and operational status, and calibration elements. Embodiments can support interactions with a system or subsystem of interest (SSoI) such as, e.g., secure and remote configurable setting changes to a system of interest or equipment within, e.g., a ship. An embodiment can include wearable systems such as, e.g., a HMD. The HMD can include one or more controller elements and/or processing sections including a plurality of processing instructions operable to operate the HMD and its sub-components such as e.g., command input interface using one or more user command input (CI) interfaces (e.g. voice, keyboard, or motion/gesture recognition) using secure user authentication systems (SUAS). HMD machine vision and pattern recognition systems can be used to visually identify a SSoI using the SUAS with a local or remote data storage, displaying a 3D model(s) of the SSoI on a visual interface of the HMD over the SSoI using an augmented reality (AR) control section, communicating with SSoI or its monitoring system to obtain a plurality of SSoI data with the SAUS and displaying one or more of the SSoI data in relation to the 3D model on the HMD visual interface. The HMD can obtain and display a comparison of a plurality of SSoI data including configurable parameters with one or a plurality of predetermined SSoI settings based on one or a plurality of operational conditions, receiving a CI request for a change of at least one configurable parameter of the SSoI, and enabling an authentication of the CI request with user access data at the local or remote data storage system.
Additional features and advantages of the present invention will become apparent to those skilled in the art upon consideration of the following detailed description of the illustrative embodiment exemplifying the best mode of carrying out the invention as presently perceived.